Skilled Messaging

This has been the longest time that I've gone between updates. It is understandable though the last few months have been full of challenges and new experiences. For one thing, I'm actually engaged to my wonderful Fiancée and starting to plan a wedding. Work has been taking up a lot of my time as well, and since I can only work on this project outside of work hours, it makes it challenging to get a lot of work on this project done. 

But this is a rather large update as well. As I mentioned in my last blog entry, I wanted to build something for my assessment piece (the main part of my project) and something that is school-related but not assessment related. So for this update I release exactly that: Skills for the assessment piece and Class Messages for the non-assessment piece.  Let's go over them individually.

Assessment Skills

When we talk about assessing someone, especially in a school setting, what do we really mean? Let's give a definition that will work in our LMS environment.

Assessment means assessing the quality of work a student can demonstrate about a certain skill, whether knowledge or character, based on a concrete scale that is transparent to all parties.

Let's dissect this a bit. First of all, there is no mention of assessing knowledge. This is, in fact, impossible, as we are not yet able to read people's brains. instead we can determine what knowledge a student is able to demonstrate.  This is a key distinction that needs to be stressed. There is no way to know if someone truly knows or understands something. We can only assess what a student can demonstrate about a subject, either through words or through actions.

The second part talks about skill of knowledge and character. There are different type of skills, which I am dividing into 2 categories: knowledge skills and character skills. Knowledge skills can be demonstrated by showing work that supports knowledge about a certain subject.  These skills build upon each other, meaning that it's hard to progress without mastering the previous material. In math terms, it's hard to do algebra if you can't demonstrate your ability to do multiplication, in English, it will be hard to write a paper if you can't demonstrate correct grammar usage. We want to keep track of all the previous work on these skills because it is the only way to catch any deficiencies early on, so if we know a student has a problem demonstrating correct sentence structure, there's no point in pushing them to do more advanced material as the foundation is not really there.

The second type of skill is what I call character skills that are more related to being a good human and student learner. They're things that we want to push our students to do because it's the right thing ethically. These skills are usually school-defined, such as “Promotes understanding in conflict resolution” for a progressive school, or “follows the tenants of morality” in something more conservative. These skills are heavily promoted in schools but not apparent in the assessment, mainly because most faculty don't think of assessing these things, rather they do it in an unofficial, “good job” sort of way. My argument is that it should be something we assess. If a student demonstrates “following the tenants of morality” in a paper they wrote, they should be assessed on these skills.

Next up in the definition is that the assessment should be based on a concrete scale. This is essentially a call that every assessment should be clear about why they're being assessed the way that they are being assessed. Now I'm not trying to reinvent the wheel here, education has long had a solution for this which is rubrics. In most cases a single rubric is used to grade an assignment. The rubric will have multiple sections with a point-award system for each section, something like this.

Rubrics are the correct way of assessing things, but I'm  of the opinion that they should be used differently in a computer system. These rubrics and the theory behind them were constructed in a time when computers and access to computers was limited and the capabilities of those systems are not what they are today.  What we end up having today is every teacher should have a rubric, but there's no effort to check this, or to have it for every assignment. There is also the fact that the rubrics are ad-hoc, with each teacher having their own version of one for each assignment. My proposal is that the rubric should be attached to the Skill, so each skill has a standard rubric on how it should be assessed. This way, when the teacher creates a Leaning Demonstration, the first thing that they select are the Skills that they will be assessing and the assessment criteria will already be attached to Demonstration, making assessment standardized based on the skills, not on whim.

The last part of definition is perhaps the one that we're lacking the most as educators, and that is assessment transparency. Very few educators show their students the rubric that they will be using for every assignment given. Most of the grades that the students get back are just a score or a letter with marking on what they did wrong. The students should be tailoring their demonstrations to what they will be assessed on, which should be clearly defined.  The idea is that the teacher would select the Skills that they will be assessing a demonstration, those skills will come attached to rubrics on how to assess them and that rubric will be front and center for the student as a constant reminder on what they're supposed to demonstrate.  Once the student submits their work, the assessment is the teacher selecting which options in the rubrics match the correct assessment, which will then generate a “grade”. The grade is not really important, it's there mainly because that's what colleges want, we will instead be pushing the real assessment as how the student was assessed in the rubric. The first thing that the student should see is not the grade, but the assessment of each rubric of the skill they are demonstrating knowledge in along with comments supporting the assessment from the teacher.  The overall grade should be optional.

Do Teachers Have Any Say in Overall Grade?

The description below might seem as it is trying to remove the assessment criteria from the hands of teachers. This couldn't be further from the truth. This system has the advantage of standardizing what skills the school feels that the students should be assessed in, but it cannot know how effective a learning demonstration is towards one set of skills. It also does not tell us how important a single demonstration is to the overall understanding of the class. What we need is for educators to define criteria for the overall assessment based on how they run their classrooms and their method of teaching and assessing. In some courses or for some teachers, the best way to demonstrate learning is through quizzes and tests and traditional repetition, this is especially true in subjects like math, while in courses such as English, papers or even oral presentation might be more important towards the overall success of the class. What we need is for the teachers to tell us two things: 1) What is the relationship between the demonstration and the skill it is seeking to demonstrate and 2) what is the relationship of the learning demonstration and the overall success in class. Both of these goals are achieved through criteria and weights.

So, when a teacher is creating a Learning Demonstration to assign to students, they first pick what skills they're seeking to assess from the students. Once selected, they then give weights on how heavily the demonstration relies on that skill. For example, if we're assigning a worksheet with addition problems, we might select a skill that deals with addition of double-digit numbers. Perhaps we also throw in a tricky problem or two that deals with carrying so we add the other skill that deals with adding with carrying. Now, most of the problems deal with assessing the first skill but we have two problems dealing with the second skill, so if there are 10 problems, we might want to assign a weight of 8 to the first standard and 2 o the last, meaning that we're assessing the first standard 8 times and the second twice. This is not always easy to do, so we need something better than numbers to represent this to the teachers, but functionally this is how it should work.

Once the Skill Assessment has been built the teacher is now required to place the demonstration in context with the rest of the class. Teachers do this by assigning Demonstrations to teacher-defined criteria that determines the “overall success” in the class. This means that the teacher gets to build how their class overall success is assessed. This is done by creating criteria that make up an overall grade calculation. Each teacher will define this criteria, which could be as easy as “Quizzes”, “Tests”, “Learning Demonstrations”, or as esoteric as “Character Development”, “Learning Comprehension”.  Each criteria is then given an overall percentage of the class which will be used in the final grade calculation. This way a teacher create a simple set of criteria weights, such as:

Then each Learning Demonstration is assigned to a Criteria with a weight, which is how the overall grade is calculated.  This is separate from the assessment, which is actually done through the rubrics; We need the grade because we live in a world of colleges and transcripts where grades are, in  opinion, incorrectly valued.

Creating Skills

Now that we have a handle on the direction that we're going, lets start won what's actually built: skills. As mentioned above, there are two types of skills, Character Skills and Knowledge Skills.  Knowledge skills seek to assess skills related to the subject being taught, while Character Skills are ethical and moral skills that we would also like to assess. This can now be done in the “Skills” submenu under the “Class Management” main menu. This will load a “folder-like” structure where all skills are organized.  

Because I wanted to have data, I went ahead and imported all of the California standards into the system. Now, an interesting thing about the California standard, which I adapted my system for, is that each standard is defined with different categories and designations.   This means that a single standard looks like this:

Note at the top that it is categorized under “English Language Arts” with a designation of “Content Area” and it is also listed under the “category" designation under the category “English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects (K–5)”. It should be noted that each subject has a different organization and category system, which makes each subject unique in how it is categorized. Because of this, every skill item can be tied to a different category through a unique designation.  This makes the Skill appear under multiple categories in the folder structure, so that teachers can approach looking for skill in different ways.

Every skill, Knowledge and Character, have certain fields that are the same. They have a designation, a full name (optional) and a description of the standard. They can be linked to one or more levels (grades) and to multiple categories through designations. They also have a rubric attached to them, which at this point is just random data that is generated through a seeder, but all rubrics can be built in the system. The main difference between Character and Knowledge Skills is that Knowledge Skills are attached to a School Subject. This will then facilitate showing only the appropriate Skills for the call being taught. Later, each standard will also be able to be attached to a course for better skill distribution to the teachers.

Class Messaging

So that last section took quite a bit to write, so I'll probably be making this smaller. The second feature of this release is the Class Messaging System, which I described in my previous post as such:

The second feature that I will be working is a self-contained feature that I plan to be done with I'm currently debating which feature I would like to add on, although I'm leaning towards the communication piece. The communication piece will be part of the class management but separate, as it is not something that you will be able to turn off. Rather, it will be used on a per-class basis as a communication tool between the teacher, student and parents. This tool was used with great success at my original school, and I haven't seen something quite like it. The idea is that all class conversation should include the parent by default.

I built this and a bit more, so let's go over it. First, as mentioned in my description, this is not a class only feature. It is not a feature that you can turn off, instead it is an integral part of the system. This automatically creates a sort of “chat-room” for every student in every class that the student, the class teacher, the parents of the student, and certain specific administrators can access. This chat is displayed differently depending on how you're viewing it, and it is accessible from the chat bubble on the upper-right hand corner.

From a teacher's perspective, the chat will load all the classes that they're responsible as a select box and then display the roster of student for each class. The teacher can click on the student and participate in the chat by typing messages. Each time a message is typed, a broadcast event is sent, so users will get real-time toasts of new messages.

The Student chat view is much simpler, since it will only list the current classes they're taking. It should be noted that there is also a new school setting, which will let the admin determine whether the chat lasts year-long or term-long.

The Parent chat view is almost identical to the student one, the only difference being that the parent can select the child that they're viewing, in cases of multiple students at the school.

The final view is the Admin chat view. Now this one is different as it ties in to another system, the “Student Tracking System”. For this release I started to develop the skeleton of this new system, but it is mostly unused. The idea behind this system is that there are certain staff/faculty members that can be given the Student Tracker role which allows them to “track” students. The goal of this system is to be able to assign students that might need help or assistance to staff that will track their progress. This can be done because the student is in academic trouble, or it can be done as a school-wide system that every student gets a dean in charge of keeping up with them.

Tracking students, as of right now, gives you admin access to their chats. As a tracker, you will see a similar view as the parent, but instead of selecting a child, you would select a student that you're tracking. The system will later be built to include an admin panel where a tacker can see a student at a glance, and be able to set “alarms” once certain things happen, such as a student forgetting an assignment, or their grade slipping below a threshold.

The Next Release

My goal remains the same: one feature related to assessment, one related to something else in the LMS. I still consider the assessment piece more crucial and purposeful, as that is the main focus of this this project. The next logical piece in the assessment would be the actual creation of Learning Demonstrations. This will be it's own feature that would allow teachers to create and post Learning Demonstrations, and for students to be able to view new and old Learning Demonstrations and interact with them in a limited fashion. 

Along with this assessment feature, another thing needs to be addressed that is directly related to it, class views. At this point, the class is only viewable by students and teachers. This needs to change to also be able to be viewed by parents and admin. This will created some issues that need to be addressed as how and when you view classes. We also need to make rules for viewing classes as both a parent and teacher, as well as admin viewing classes and admin tracking students.  This will be part of the assessment feature, as it will include a way to view, create and interact with Learning Demonstrations.

As for the non-assessment feature, I've decided on ID's.  The fact is that we should be obfuscating the user id's of people. Also, if you're creating physical student (or faculty) ids, you would like to include more “standard” numbers, maybe with a set number of digits, but not like “1” or “134”.  I would like the ids to be placed on every person in the system, and that every person has an actual “soft" ID Card that can be accessed by the users. This ID should be admin-created and will eventually be tied into the user check/checkout system as well as other systems. The id should also be able to be used in Google/Apple Wallets, although I don't want to sign up agreements, so I will make them optional.

I will update when it's done.