Assessment Module Assessment Module Overview Relevant Definitions Please review Assessment Function. Assessment Code PEMS tracks any number of assessments and relates each assessment to programs, subjects, curriculum selection, grade level and grade span. The assessment are defined in the assessment code table, and they may be linked to any combination of the above records.  The assessment code is a plan to asses a student for an indicator, it is not the actual student result. Assessment Type The assessment Type field has the three options for the different types of assessments that PEMS supports. Level Equivalent - assessment determines what grade level the student is actually performing at in the subject Percentile Rank - assessment determines the ranking against a larger population of students at the same grade level Competency - assessment is recording a student has completed a required competency for the grade span Assessment Code and the Subject Each assessment code record may only link to one subject, assessments codes may not be defined for multiple subject areas in one code.  When students take one exam that tests the student on multiple subjects, the school must create an assessment code record for each of the subjects the single test addresses.  When the test results are records, the user must create multiple assessment result records for such a test.  In General, tests that cover multiple subject areas provide individual scores for each of the tested areas.  Because an assessment code record only allows one score, multiple assessment code records are needed to record all available data. The school does have the option to define subjects as very high level, in doing this one one assessment code record may be created for the high-level subject. Marks Matrix The marks matrix table allows the the administrator to interpret the raw scores of each assessment in a consistent manner.  Multiple marks matrix records are created for each assessment code record are used to specify a range of scores.  Each mark matrix record specifies a low mark and high mark. If the raw score stored in the assessment result record falls inside the range of a specific marks matrix record, then the parent assessment result record will read the 'normalized' score from the mark matrix record and update the assessment result record. Level Equivalent For Assessment codes with type = level equivalent, the normalized score will be the level equivalent field from the matching marks matrix record.   Recall the definition of Level Equivalent: Students are assessed for their progress towards acquiring the knowledge and skills of a high school graduate, a student who completed 13 years of schooling.  Given enough time and effort, all students can reach this standard.  Level equivalent is an approach that assesses if a student is progressing faster or slower towards their graduation.   If the student is progressing slower, they are behind their grade level, if the are progressing faster, they are above their grade level.  PEMS allows users to define the grade level equivalent for each score. As I understand it, this means that each school head can determine personally what it means to say be at a 6th grade level. Normally, how is the national grade level competency determined. Would most school heads go off a national standard determined by the Department of Education? Percentile Rank For assessment codes with type = percentile rank, the normalized score will be the percentile rank score from the matching marks matrix record. recall the definition for percentile rank: Percentile ranking is another method of assessing student performance.  With this method, students are ranked against their peers in a larger population, either state-wide or nationally.  Traditionally, a student's grade in a course can be considered percentile ranked, for example only 10% of students should receive and A in the class.  Percentile Ranked assessments are similar but across a larger population of students. For these types of assessments, PEMS stores a raw score and a percentile rank in the assessment result record. Competencies The marks matrix table is not used for competency based assessments since these assessments are either passed or not passed, they are not comparative in nature. Assessment Result Calculations Assessment Results Calculations An assessment result has limited value if the administrator can not see a trend over time in how the student is performing.  Therefore, the difference in academic performance between a previous assessment result must be calculated, but the difference in time must also be calculated in order to obtain a trend. The following terminology is used when making these calculations. When there are multiple assessment result records associated with the one assessment code, the student is being tested multiple times on the subject area.  The set of assessment results records creates a picture of how the student performs over time. Each of theses assessment result records can "look back in time" and calculate improvement since the last assessment result record associated to that same assessment code. These calculations are performed on records in the Assessment Results Table - TBD in the Students Module. They are not performed on any table in the Assessments Module. Level Equivalent Assessments Calculations PEMS uses hundreds/tens place pattern for its Level Equivalent scoring.  The grade level in in the hundred's place, and the month is in the tens place. For example 840 is eight grade 4th month.  Withing a grade year, this is a quick visual reference for improvement, 800 compared to 860 represents 6 months of academic progress. Parents and student readily understand this system and it is easy to explain.  A student starts a grade at 800, then increased by ten for every month in school. After the ninth month, the student has achieved 890.  Then the student takes summer off and starts 9th grade at 900.  890 is the same as 900 under this system.  This issue becomes important when calculating grade level differences in grade level achievement in some cases. The level equivalent assessment measures the academic performance in terms of grade levels.  Regular, expected grade level performance means that a student' level equivalent increases 1 grade level per 1 academic year, so a regular student's average improvement rate is 1. For example, a student's  assessment result record is linked to the assessment code record names "IXL basic arithmetic".  At the beginning of the term, September 4, 2023, the student scores 720.  This score means 7tth grade second month. An assessment result record is made with date and score, linked to the assessment code, "IXL basic arithmetic" Four months later on January 5, 2024, the student scores a 840.  This score means 8th grade 4th month. 4-SEP-2023    720    IXL Arithmetic 5-JAN-2024    840    IXL Arithmetic The difference in scores is: 840-720 = 120 in improvement = 1.2 grade years of progress The difference in time is 5-Jan 2024 - 4-SEP 2023 = about 19 weeks  = 1 term or half a school year. The student's improvement rate is 1.2/0.5 = 3.4 grade levels per year (or 3.4 times the expected rate) Naming Convention AP stands for Academic Performance, in the example above, AP is 720 and 840. The letter "D" stands for difference, just like delta or the 'd' in calculus. DT stands for the Difference in Time frame.  Time is measured in terms, grade spans, and grade levels. In the example above, the difference in time is 19 weeks, but the difference in time frame is 0.5 academic years. Examples AP = Academic Performance, D stand for Difference, therefore: DAP = different in Academic Performance values between two assessment result records that are associated to the same assessment code record.  In the example above, DAP = 840 - 720 = 120. DT = the difference in time as measured in academic years,  therefore: DTterm = the difference over a term DTGrade = the difference  over the year DTSpan = the difference over the a grade span Rate Calculations DAPDTerm  is the difference in academic performance divided by the corresponding difference in time.  This is the near term outlook, looking at how the student is progressing term by term. DAPDTGrade is the difference in academic performance divided by the corresponding difference in time.  This value looks at how the student performed in a giver academic year. DAPDTSpan is the difference in academic performance divided by the corresponding difference in time.  This value takes the widest look at progress over a longer horizon. Time is always measured in portion of academic years At any one point in time the student may not have completed a whole term, a whole grade level, or a whole grade span.  Time is  between assessment results is always measured in units of academic years.  The DT or "delta time" is derived from the difference of the assessment result in days divided by the number of days in an average academic year. Selecting the correct DAP - which assessments results to use A student may take an assessment at anytime in the academic school year.  When calculating  time frame differences, PEMS selects the most relevant historical assessment record. For the term time frame, PEMS finds the oldest assessment within the current term. For the Grade time frame, PEMS find the oldest assessment result at the students current grade level. For the Grade Span time frame, PEMS find the oldest assessment result within the grade span. For a new student that has only two similar assessments, the all values above are the same.  The values are only different as more assessments are taken. In the second term the DAPDTGrade uses the very first record, while the DADTerm uses a more recent record to make the calculation. Percentile Rank A percentile rank stores a student's score relative to a wider population of students.  However, a raking is always against a similar grade level. For example, a student may be in top 90% of sixth graders if in sixth grade. Because percentile rank is still measured against grade level, the calculations are no different. The student in one term may increase from 60% rank to 65% rank in 0.5 years. The student's rate of progress is (65-60)/0.5 years or 10% climb per year.  For the grade span calculation, If the student climbed from 60% rank to 85% rank over 2.5 years, then the rate is (85-60)/2.5 or 10% per year. Note: Because the student transitioned from being ranked against 6th grades to being ranked against 8th graders is not material since the student's own grade also advances. Calculations Workflow When ever a new assessment result record is entered, each result is associated to an assessment code. PEMS searches for prior assessment result records, and calculates the difference in academic performance  between assessment result records and the difference in time frames.  Then PEMS calculates all the various parameter explained herein. There is no need for this algorithm to be triggered on the event, the algorithm can run nightly to update all assessment result records. Once entered, these values are never recalculated.Assessment Codes Table Marks Matix An assessment code record is created for every time of assessment presented to students.  The school should create separate assessment codes for different test vendors as no vendors will every be completely the same.  A unique code is given to each type of assessment.  When a student takes that assessment, the results are recorded and associated with the assessment code.  All student scores per code may be reported on. Data Relationships There are multiple tables that may be linked to an assessment code record.  Not all are required. Subject is required as the school should not be assessing students in subjects it does not teach.  If the student takes a single test with multiple subjects at one time, multiple code records will need to be creates, each with a unique subject.  Then when the students assessments results are input, multiple student assessment results will also be required. Assessment codes may be linked to plans to determine the effectiveness of the plan.  A plan may have many assessment codes. Assessment codes may be further linked to a curriculum in that plan if the effectiveness of the curriculum should be assessed. Assessment codes may be linked to a grade level for example if that assessment is an important 'gate' before embarking of a course of study. When a student takes an assessment, the results are input into the student assessment code table, one student will have multiple student assessment results records associated with one assessment code over time.  For example, a student may take the SAT multiple times, these scores may be input and linked to the same SAT assessment code record. Fields Assessment name - should contain the name of the vendor if third party test is uses. Code - an short a unique abbreviation for the assessment. Type -how the student is being measured. Level Equivalent - the student score reflects where they stand against what the larger world expects from a student at that grade level. Percentile Rank  - the test ranked the students against other students in the same grade level. Competence - an yes/no determination if the student can perform the to the standard. Normalize Scores - toggle this to yes to use the Marks Matrix table to normalize the student's raw test scores to fit one of the selected types. Action Items JAN 2024 NV | ASSESSMENT | Implement Mark Matrix logic to normalize scores for assessments that are not already level equivalent.Marks Matrix PEMS provide the capability to normalize any test score into the types provides, level equivalent and percentile rank. If a student takes a test and earns a raw score of 525, what does that mean?  The school administrator can specify the ranges of the scores that would map to a grade level equivalent or a percentile rank. Data Relationships There are many Marks Matrix records associated to the Assessment Code record.  The student's assessment results are linked to the code record, and will use the information in the Marks Matrix table to convert the raw score the a level equivalent of percentile rank. Fields Low Mark - the low value in the range - user input High Mark - the high value in the range - user input Mark Range - calculate the different between high and low mark Level Equivalent - 3 digit number with grade level in hundreds place, month in tens place - user input Equivalent Year - the grade year portion of the level equivalent - calculate Equivalent Month - the grade month portion of the level equivalent - calculate Percentile Rank - the ranking that a score in the range would be equivalent to. Action Items Jan 2023 - TBDCompetency Assessment Integration Relevant Topics Please review: Assessment Function Tracking Competencies allow schools to ensure students  meet all their objectives. A student may score very high in reading competency, achieve a B in an English class, but has never turned in a reasonably well written essay.  Alternatively, a student may archive a B in History with excellent knowledge of the Roman Empire, but with no domain knowledge of World War II.  Good grades may not catch a lack of competency or domain knowledge in areas important the philosophy of the school requires. Moodle competencies are specified in the activity settings, internal to a Moodle course by the course creator. Each activity may be tagged with pre-defined competencies. When the student completes the activity, they earn the competency.  The competency list is created in the admin panel. In Moodle, a student can be assigned a learning plan, which is a group of competencies that may span multiple courses.  The student can view their progress towards the learning plan in their profile. Moodle allows the admin to define Learning Plans, which are sets of competencies.  Learning Plan templates are use to assign learning plans in bulk to students.  Templates may be assigned to a Cohort of students. In real life, a cohort is a set of students at a specific grade level. All ninth graders in 2023 form a set of students, as they move through the various grade levels, we refer to those 2024 ninth graders as a cohort. In Moodle a cohort is a set of students and a teacher(s).  Cohorts are primarily used in Moodle for batch enrollment.  For the use case that all students in 4th grade should be enrolled in 4th grade math.  In this case a Moodle user would define a cohort of 4th graders, then assign the cohort to a specific course.  However, using cohorts for enrollment is not useful in PEMS because each student is following a unique program. The unique program is managed in the SIS.  In PEMS, all fourth graders are not enrolled in fourth grade math. However, cohorts are still useful for reporting.  So in PEMS, Moodle Cohorts are used to identify a set of students under the supervision of a particular teacher (proctor).  In this use case, students can be at any grade level since PEMS is designed for mixed grade-level student management.  Unfortunately, Moodle Cohorts can not be used to block a teacher from seeing other student's work who are not in their own cohort. In summary 1) Moodle Cohorts are a set of students and their teachers (proctors)  2) the students in the set can be any grade level  3) Cohorts are useful for reporting 4) All teachers can see all students in any cohort. Proposed Integration of Competencies and Assessment Codes In the SIS, assessment codes may be defined with type = competency.  This information is entirely driven by students completing activities (assignments)  in Moodle.  Competency tracking requires tight synchronization between PEMS and Moodle and is not useful unless Moodle has been configured to use competencies and Learning plans, as students may only demonstrate they meet the competencies through completing certain assignments in various Moodle courses. Competencies are defined in Moodle in the admin panel. All Moodle competencies have an ID Number field, which is unique. For synchronization between SIS and Moodle, the admin must define a corresponding Assessment Code record and set the Code field to the exact value as the Moodle Competency ID Number.  This is similar to how the curriculum table is used to reflect Moodle courses in the SIS. The Assessment Code records in PEMS must have the exact same text string as the Competency Code field in Moodle. When a Moodle Activity is configured to demonstrate a competency, Moodle marks this competency complete in Moodle. Moodle makes a yes/no determination on competency, there is no score.  A set of Moodle activities can also be configured to complete one competency. Update workflow The event trigger for Moodle to update the SIS takes place whenever a student earns a competency.  Moodle completes the competency for a student, an assessment result record need to be created, linking the assessment code record in SIS to the student, and recording the date the competency was earned and the yes/no outcome from Moodle. In SIS, the list of competencies are stored as Assessment Code records with "Code" set equal to Moodle Competency "ID number" In Moodle the exact same list of competencies are stored as competency records. In Moodle activities are linked to competencies. In Moodle Students are assigned a set of competencies, which may be assigned based on cohort. Moodle determines if the competency has been achieved When achieved, an assessment result record is created for the student with date of achievement no records are created in SIS if the student has failed the competency in Moodle Hierarchy of Competencies Moodle offers unlimited hierarchies in competencies, when all sub competencies are completed, Moodle checks off the higher level competency as completed.  PEMS does not duplicate the hierarchy and has only a flat file of competency records.  If the school wants to reflect the hierarchy in Moodle in PEMS, the best approach is to use an agreed convention in the name field of the Assessment Code record.  Regardless of how Moodle checks of a competency when it is checked off an Assessment Result record is created in PEMS. Assessment Code records may be linked to subject and grade level.  Learning Plans In Moodle and Programs of Study In Moodle, a student can be assigned a learning plan, which is a group of competencies that may span multiple courses.  The student can view their progress towards the learning plan in their Moodle profile. Learning plan templates, the set of competencies that comprise the student's assigned learning plan, have a unique ID number field. All learning plans should be developed to support the objectives of the program of study.  Each plan in PEMS should be linked to the learning plan in Moodle.  The learning plan in Moodle tracks he competences required by that plan. Proposed Integration of Learning Plan and Programs of Study The SIS's Program of Study table should add a field called "Moodle Learning Plan". The Moodle learning plan field value in the plan record stores the exact match of the learning plan template ID number in Moodle that reflects the objectives of the plan. When the plan is assigned to the student, this will also trigger Moodle assigning the learning plan to that student based since the plan is linked to the Moodle learning plan template ID number. As the student completes the Moodle courses in the plan, the student will earn the competencies, and Moodle will then create assessment result record to demonstrate the competencies. Data Structure Conflict When assessment code records are linked to Plans, they may see the learning plan template ID number.  An assessment code record may only be associated with one plan.  This differs from Moodle where the same competency may be linked to multiple learning plans.  Consequently, multiple assessment code records must be created in SIS where the is only one competency ID in Moodle.  The name of the competency should roughly the same for the same competency in Moodle. The assessment codes do not need to be linked to the plans to 1) make records of completed Moodle competencies or 2) auto-enroll a cohort with the program of study's specified learning plan. However if SIS is to track and report of student progress towards the competencies, then the total set of competencies must be known, and the gaps between completed and incomplete reported on.